Skip to content

gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants#148081

Open
corona10 wants to merge 24 commits intopython:mainfrom
corona10:gh-145742-work
Open

gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants#148081
corona10 wants to merge 24 commits intopython:mainfrom
corona10:gh-145742-work

Conversation

@corona10
Copy link
Copy Markdown
Member

@corona10 corona10 commented Apr 4, 2026

@corona10
Copy link
Copy Markdown
Member Author

corona10 commented Apr 5, 2026

@Fidget-Spinner @markshannon PTAL when you have times :)

@corona10
Copy link
Copy Markdown
Member Author

corona10 commented Apr 8, 2026

I’ve just resolved all the conflicts. Gentle ping to @markshannon and @Fidget-Spinner.

@markshannon
Copy link
Copy Markdown
Member

Thanks for doing this.

There is work underway to remove the many variants of _LOAD_CONST_INLINE_BORROW that also pop, swap or rotate.
Once that work is done, this PR should become much simpler.
I'd recommend leaving this for a week or two before attempting to resolve conflicts.

@corona10
Copy link
Copy Markdown
Member Author

corona10 commented Apr 9, 2026

Got it

@corona10
Copy link
Copy Markdown
Member Author

@markshannon I've resolved conflicts with the latest main and simplified the pre-tagging/untagging to be path-based (auto-tagging in add_op() when the opcode is a BORROW variant). PTAL.

Copy link
Copy Markdown
Member

@markshannon markshannon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tested this with Py_STACKREF_DEBUG set?

Comment thread Include/internal/pycore_stackref.h Outdated
#define PyStackRef_FromPreTagged(ptr) _PyStackRef_FromPreTagged((uintptr_t)(ptr))

/* Tag a PyObject pointer as a borrowed operand for BORROW variants. */
#define PyStackRef_TagBorrow(ptr) \
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't it need defining for Py_STACKREF_DEBUG as well?
Can you make this an inline function, for type safety?

Comment thread Python/executor_cases.c.h
PyObject *ptr = (PyObject *)CURRENT_OPERAND0_64();
value = PyStackRef_FromPyObjectBorrow(ptr);
_PyFrame_SetStackPointer(frame, stack_pointer);
value = PyStackRef_FromPreTagged(ptr);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to whitelist PyStackRef_FromPreTagged as non-escaping

@corona10
Copy link
Copy Markdown
Member Author

Have you tested this with Py_STACKREF_DEBUG set?

FYI, it's already broken see: #148718

@corona10 corona10 requested a review from markshannon April 20, 2026 16:35
@corona10
Copy link
Copy Markdown
Member Author

@markshannon gentle ping?

@markshannon
Copy link
Copy Markdown
Member

This looks fine, once updated to main.

The problem I have with it is the lying to the type system and tagged pointers.
The fix would be to differentiate between object pointers, other pointers, stackrefs and 64 bit integers in the DSL. But that's a lot of work for this small change.

So, can you either:

  • Make a new issue make those changes to the DSL in the future, and then we can merge this, or
  • Make the DSL changes in this PR

Comment thread Include/internal/pycore_stackref.h Outdated
Comment thread Python/bytecodes.c

tier2 pure op(_LOAD_CONST_INLINE_BORROW, (ptr/4 -- value)) {
value = PyStackRef_FromPyObjectBorrow(ptr);
value = PyStackRef_FromPreTagged(ptr);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that we always cast /4 values to PyObject * even when they aren't pointers.

Comment thread Python/bytecodes.c Outdated
Comment thread Python/optimizer_analysis.c Outdated
@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented Apr 30, 2026

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@corona10
Copy link
Copy Markdown
Member Author

corona10 commented Apr 30, 2026

Make the DSL changes in this PR

I prefer this one, since this PR is not urgent thing :)

@corona10 corona10 changed the title gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants [WIP] gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants May 1, 2026
@corona10 corona10 changed the title [WIP] gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants gh-145742: Pre-tag operands for _LOAD_CONST_INLINE_BORROW variants May 1, 2026
@corona10 corona10 requested a review from markshannon May 1, 2026 10:04
@corona10
Copy link
Copy Markdown
Member Author

corona10 commented May 1, 2026

@markshannon
I’ve added a ^ marker to annotate the pretagged cache effect.

I’d like to know whether this is the kind of DSL you had in mind, or if you would prefer a different grammar. I would be happy to follow your suggestion if you have one.

PTAL.

    op ( LOAD_CONST_INLINE_BORROW, (ptr/4^ -- value) ) {
        value = PyStackRef_FromPreTagged(ptr);
    }

@corona10
Copy link
Copy Markdown
Member Author

corona10 commented May 1, 2026

FYI, the current Ubuntu CI failure is not related to this change; it's a Canonical infra issue that they are handling at this moment. https://x.com/ubuntu/status/2050112955132297652

@corona10
Copy link
Copy Markdown
Member Author

corona10 commented May 1, 2026

DSL Summary

  • The ^ marker on cache slots (e.g. ptr/4^) tells the DSL the operand is pre-tagged, so consumers declare uintptr_t + read_u64 instead of lying with PyObject *.
  • Producers (add_op, convert_global_to_const) now call a generated _PyUop_PrepareOperand0 dispatch keyed on the same marker, so no more hardcoded _LOAD_CONST_INLINE_BORROW checks, future ^ ops are wired in by DSL alone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants